Spoken affect classification : algorithms and experimental implementation : a thesis presented in partial fulfilment of the requirements for the degree of Master of Science in Computer Science at Massey University, Palmerston North, New Zealand

Machine-based emotional intelligence is a requirement for natural interaction between humans and computer interfaces and a basic level of accurate emotion perception is needed for computer systems to respond adequately to human emotion. Humans convey emotional information both intentionally and unintentionally via speech patterns. These vocal patterns are perceived and understood by listeners during conversation. This research aims to improve the automatic perception of vocal emotion in two ways. First, we compare two emotional speech data sources: natural, spontaneous emotional speech and acted or portrayed emotional speech. This comparison demonstrates the advantages and disadvantages of both acquisition methods and how these methods affect the end application of vocal emotion recognition. Second, we look at two classification methods which have gone unexplored in this field: stacked generalisation and unweighted vote. We show how these techniques can yield an improvement over traditional classification methods

Latent variable modelling of user interaction in image retrieval

Cette thèse étudie les modèles à variables latentes sur les interactions utilisateur avec l'objectif d'améliorer la recherche d'images. Les historiques de recherche, appelés query logs, où l'interaction entre les utilisateurs et le système de recherche est enregistrée, contiennent souvent les indications d'intention sous la forme de jugements de pertinence donnés sur les documents dans le contexte d'une recherche. Selon la nature du système de recherche et de l'interaction qu'il permet, ces jugements peuvent être explicites ou implicites, et, une fois agrégé un grand nombre des recherches effectuées par de nombreux utilisateurs, ils peuvent être exploités pour améliorer divers aspects du système de recherche. Cette thèse propose un modèle des historiques de recherche, le Modèle de Pertinence Utilisateur, où les jugements de pertinence sont issus d'un processus génératif par lequel l'utilisateur juge (soit implicitement soit explicitement) un document comme pertinent s'il partage un degré de recouvrement avec la requête en termes de concepts, et non pertinent dans le cas contraire

Hierarchical long-term learning for automatic image annotation

This paper introduces a hierarchical process for propagating image annotations throughout a partially labelled database. Long-term learning, where users' query and browsing patterns are retained over multiple sessions, is used to guide the propagation of keywords onto image regions based on low-level feature distances. We demonstrate how singular value decomposition (SVD), normally used with latent semantic analysis (LSA), can be used to reconstruct a noisy image-session matrix and associate images with query concepts. These associations facilitate hierarchical filtering where image regions are matched based on shared parent concepts. A simple distance-based ranking algorithm is then used to determine keywords associated with regions

TagCaptcha : annotating images with CAPTCHAs

Image retrieval has long been plagued by limitations on automatic methods because they cannot reliably extract semantic data from low-level features. The result is that users must formulate awkward and inefficient queries in terms these systems can understand. Humans, on the other hand, have the ability to quickly and accurately summarise visual data. This dichotomy, named the semantic gap, is a fundamental problem in image retrieval. We aim to narrow the semantic gap in a typical retrieval scenario by motivating users to provide semantic image annotations. We propose a system of collecting image annotations based on the need for human verification on the web. Similar in principle to work by von Ahn et al. [2, 3], the idea is to exploit the requirement of users to pass tests in order to incrementally annotate images

Topic modelling of clickthrough data in image search

In this paper we explore the benefits of latent variable modelling of clickthrough data in the domain of image retrieval. Clicks in image search logs are regarded as implicit relevance judgements that express both user intent and important relations between selected documents. We posit that clickthrough data contains hidden topics and can be used to infer a lower dimensional latent space that can be subsequently employed to improve various aspects of the retrieval system. We use a subset of a clickthrough corpus from the image search portal of a news agency to evaluate several popular latent variable models in terms of their ability to model topics underlying queries. We demonstrate that latent variable modelling reveals underlying structure in clickthrough data and our results show that computing document similarities in the latent space improves retrieval effectiveness compared to computing similarities in the original query space. These results are compared with baselines using visual and textual features. We show performance substantially better than the visual baseline, which indicates that content-based image retrieval systems that do not exploit query logs could improve recall and precision by taking this historical data into account